Home

Placeholder

Introduction

Definitions of Engagement

The roots of employee (aka work; e.g., W. Schaufeli & Bakker, 2010) engagement research likely started with theoretical expansions of forms of employee participation (see, for example, Ferris & Hellier, 1984) and job involvement (e.g., Elloy et al., 1991). This exploration extended into broader considerations of attitudes and emotions (Staw et al., 1994) and were informed by further exploration of the dimensionality of constructs such as organizational commitment (Meyer & Allen, 1991). The 1990’s saw focused development and refinement (for example, a dissertation; Leone (1995) or actual semantic reference; Kahn (1990)). Staw et al. (1994) investigated the relationships between positive emotions and favourable work outcomes, and although they do not use the word, “engagement,” their distinction between felt and expressed emotion was likely a stimulating influence upon the burgeoning interest in the engagement construct. Gallup is also widely acknowledged as an early pioneer in the measurement of the construct (see, for example, Coffman & Harter, 1999).

Kahn (1990) described engaged employees as being physically involved, cognitively vigilant, and emotionally connected. Although occasionally referred to as residing on the opposing pole to burnout (Christina Maslach & Leiter, 2008), these two constructs are currently most commonly conceptualized as being distinct (Timms et al., 2012), although not universally (Cole et al., 2012; Taris et al., 2017).

  • Engagement/Burnout: Burnout is defined as a psychological syndrome characterized by exhaustion, cynicism, and inefficacy, which is experienced in response to chronic job stressors. Engagement is understood to be the direct opposite of burnout and exist on a continuum—with engagement on one end and burnout on the other. Exhaustion (low energy), cynicism (low involvement), and inefficacy (low efficacy) are characteristic of burnout; whereas, high energy, high involvement, and high efficacy are characteristic of engagement C. Maslach & Leiter (1997) (Maslach and Leiter, 1997) (Leiter & Maslach, 2004).

  • Work engagement: Work engagement refers to a positive, fulfilling, work-related state of mind that is characterized by vigor, dedication, and absorption. Vigor is characterized by high levels of energy and mental resilience while working. Dedication refers to being strongly involved in one’s work and experiencing a sense of significance, enthusiasm, inspiration, pride, and challenge. Absorption is characterized by being fully concentrated and happily engrossed in one’s work, whereby time passes quickly and one has difficulties with detaching oneself from work (Wilmar B. Schaufeli et al., 2002) - The Utrecht Work Engagement Scale (UWES), a self-report questionnaire first developed by Schaufeli and Bakker (2003) (W. B. Schaufeli & Bakker, 2003) includes the three aspects of work engagement: vigor, dedication, and absorption. - we have to make note of how we are different from this!

  • Employee engagement: Employee engagement refers to the ‘‘individual’s involvement and satisfaction as well as enthusiasm for work’’ (James K. Harter et al., 2002) Work engagement is the mental state where employees…

  • …feel full with physical energy (Vigor)

  • …are enthusiastic about the content of their work and the things they do (Dedication)

  • …are so immersed in their work activities that time seems to fly (Absorption)

The tripartite substantive model of employee engagement is also partially informed by the definitions provided with the Utrecht Work Engagement Scale (Wilmar B. Schaufeli & Bakker, 2003).

Instrument creation

Placeholder

Intended Structure

Item generation

Content Validation

Procedure

Placeholder

Pilot

Sampling strategy (pilot)

Pilot analyses

Demographic Information

Analyses

Pilot analyses

282 people responded. We may want to further screen these 282 responses for short response durations. According to communications with Qualtrics in early December, this is the number of seconds to complete the entire survey. Currently, the “number of people who clicked on the assessment link (330)” versus valid n (282) may take care of our very low duration respondents. The shortest response duration in the 330 datafile is 101 whereas the lowest in the 282 datafile is 43. All analyses were performed via R version R version 4.1.0 (2021-05-18) (R Core Team, 2021).

Classical test theory

In addition to the below interactive plot [via plotly version 4.9.4.1; Sievert et al. (2021)], a full inter-item correlation matrix is located in Appendix @ref(corrs).

## Some items ( I find it difficult to mentally disconnect from work. ) were negatively correlated with the total scale and 
## probably should be reversed.  
## To do this, run the function again with the 'check.keys=TRUE' option

Internal consistency estimates

Condition 1 administered items within the substantive dimensions (with successive randomized blocks of Cognitive, Affective, and Behavioral items). Condition 2 administered items within the attitudinal dimensions (with successive randomized blocks of Absorption, Vigor, and Dedication items). Condition 3 stressed the substantive dimensions (with items fully randomized regardless of attitudinal association). Condition 4 stressed the attitudinal dimensions (with items fully randomized within attitudinal dimension regardless of substantive scale association, see Chapter @ref(conds) and Appendix @ref(pilot2)). All internal consistency estimates were generated via psych version 2.1.6 (Revelle, 2021). Alphas for the candidate 12-item scales were:

Dimension Undifferentiated Condition 1 Condition 2 Condition 3 Condition 4
Affective 0.87 0.88 0.84 0.88 0.87
Behavioral 0.79 0.73 0.84 0.76 0.87
Cognitive 0.81 0.79 0.85 0.8 0.87
Absorption 0.74 0.72 0.77 0.72 0.87
Vigor 0.85 0.8 0.88 0.84 0.87
Dedication 0.9 0.9 0.91 0.91 0.87

“Cell” level alphas (4 items each scale, responses collapsed across administrative conditions) were:

Cell Alpha
Affective - Absorption 0.66
Affective - Vigor 0.71
Affective - Dedication 0.75
Behavioral - Absorption 0.56
Behavioral - Vigor 0.7
Behavioral - Dedication 0.64
Cognitive - Absorption 0.59
Cognitive - Vigor 0.62
Cognitive - Dedication 0.83

Corrected item-total correlations are presented in Appendix @ref(rdrops)

Confirmatory factor analyses

We used lavaan version 0.6.9 (Rosseel et al., 2021) and semPlot version 1.1.2 (Epskamp, 2019)

Model \(\chi^2\) df RMSEA SRMR CFI TLI AIC
3-factor substantive 2159.21 591 0.11 0.1 0.64 0.62 25481.97
Hierarchical substantive 2159.21 591 0.11 0.1 0.64 0.62 25481.97
3-factor attitudinal 2318.92 591 0.11 0.1 0.6 0.58 25641.68
Hierarchical attitudinal 2318.92 591 0.11 0.1 0.6 0.58 25641.68

Multigroup analyses

  • using experimental condition as “multiple groups” (measurement invariance)

Bifactor analyses

bifactor analysis are most commonly applied in the exploration of common method variance (see, for example, Reise, 2012; Rodriguez et al., 2016). Giordano et al. (2020) provide an overview regarding past and potential applications of exploratory bifactor analysis and cite Reise (2012) as an influential impetus for the resurgence of bifactor models in general.

## lavaan 0.6-9 did NOT end normally after 820 iterations
## ** WARNING ** Estimates below are most likely unreliable
## 
##   Estimator                                         ML
##   Optimization method                           NLMINB
##   Number of model parameters                       114
##                                                       
##                                                   Used       Total
##   Number of observations                           239         282
##                                                                   
## Model Test User Model:
##                                                       
##   Test statistic                                    NA
##   Degrees of freedom                                NA
## 
## Parameter Estimates:
## 
##   Standard errors                             Standard
##   Information                                 Expected
##   Information saturated (h1) model          Structured
## 
## Latent Variables:
##                    Estimate   Std.Err  z-value  P(>|z|)
##   Cognitive =~                                         
##     Item_1             1.000                           
##     Item_2             0.516       NA                  
##     Item_3             1.090       NA                  
##     Item_4             0.353       NA                  
##     Item_13            1.130       NA                  
##     Item_14            0.838       NA                  
##     Item_15            0.798       NA                  
##     Item_16            1.098       NA                  
##     Item_25            0.988       NA                  
##     Item_26            0.909       NA                  
##     Item_27            1.080       NA                  
##     Item_28            0.999       NA                  
##   Affective =~                                         
##     Item_5             1.000                           
##     Item_6             0.656       NA                  
##     Item_7             0.475       NA                  
##     Item_8             1.242       NA                  
##     Item_17            0.933       NA                  
##     Item_18            1.372       NA                  
##     Item_19            1.044       NA                  
##     Item_20            0.536       NA                  
##     Item_29            0.841       NA                  
##     Item_30            0.381       NA                  
##     Item_31            0.724       NA                  
##     Item_32            1.063       NA                  
##   Behavioral =~                                        
##     Item_9             1.000                           
##     Item_10            1.109       NA                  
##     Item_11            0.572       NA                  
##     Item_12            1.176       NA                  
##     Item_21            1.362       NA                  
##     Item_22            1.573       NA                  
##     Item_23            0.873       NA                  
##     Item_24            0.890       NA                  
##     Item_33            0.707       NA                  
##     Item_34            0.879       NA                  
##     Item_35            1.237       NA                  
##     Item_36            0.369       NA                  
##   Absorption =~                                        
##     Item_1             1.000                           
##     Item_2         19131.383       NA                  
##     Item_3          -390.734       NA                  
##     Item_4         21074.473       NA                  
##     Item_5          2937.496       NA                  
##     Item_6         -1569.896       NA                  
##     Item_7           727.592       NA                  
##     Item_8         -1533.787       NA                  
##     Item_9          4326.028       NA                  
##     Item_10         5122.173       NA                  
##     Item_11        -1534.018       NA                  
##     Item_12        -1356.265       NA                  
##   Vigor =~                                             
##     Item_13            1.000                           
##     Item_14           -3.413       NA                  
##     Item_15            0.449       NA                  
##     Item_16           -0.517       NA                  
##     Item_17           -0.246       NA                  
##     Item_18           -1.152       NA                  
##     Item_19            0.766       NA                  
##     Item_20           -3.510       NA                  
##     Item_21            0.381       NA                  
##     Item_22           -0.278       NA                  
##     Item_23            0.324       NA                  
##     Item_24            0.093       NA                  
##   Dedication =~                                        
##     Item_25            1.000                           
##     Item_26            1.378       NA                  
##     Item_27            1.009       NA                  
##     Item_28            0.830       NA                  
##     Item_29            0.885       NA                  
##     Item_30            0.650       NA                  
##     Item_31            0.274       NA                  
##     Item_32            0.512       NA                  
##     Item_33            0.010       NA                  
##     Item_34            0.122       NA                  
##     Item_35            0.888       NA                  
##     Item_36            0.783       NA                  
## 
## Covariances:
##                    Estimate   Std.Err  z-value  P(>|z|)
##   Cognitive ~~                                         
##     Affective          0.549       NA                  
##     Behavioral         0.363       NA                  
##   Affective ~~                                         
##     Behavioral         0.372       NA                  
##   Absorption ~~                                        
##     Vigor              0.000       NA                  
##     Dedication        -0.000       NA                  
##   Vigor ~~                                             
##     Dedication        -0.082       NA                  
##   Cognitive ~~                                         
##     Absorption         0.000                           
##     Vigor              0.000                           
##     Dedication         0.000                           
##   Affective ~~                                         
##     Absorption         0.000                           
##     Vigor              0.000                           
##     Dedication         0.000                           
##   Behavioral ~~                                        
##     Absorption         0.000                           
##     Vigor              0.000                           
##     Dedication         0.000                           
## 
## Variances:
##                    Estimate   Std.Err  z-value  P(>|z|)
##    .Item_1             1.343       NA                  
##    .Item_2             0.350       NA                  
##    .Item_3             0.786       NA                  
##    .Item_4             0.124       NA                  
##    .Item_13            0.809       NA                  
##    .Item_14            0.659       NA                  
##    .Item_15            1.471       NA                  
##    .Item_16            0.760       NA                  
##    .Item_25            1.350       NA                  
##    .Item_26            0.629       NA                  
##    .Item_27            1.282       NA                  
##    .Item_28            0.763       NA                  
##    .Item_5             1.324       NA                  
##    .Item_6             1.195       NA                  
##    .Item_7             0.845       NA                  
##    .Item_8             0.688       NA                  
##    .Item_17            0.484       NA                  
##    .Item_18            0.559       NA                  
##    .Item_19            0.771       NA                  
##    .Item_20            0.780       NA                  
##    .Item_29            0.479       NA                  
##    .Item_30            1.514       NA                  
##    .Item_31            0.538       NA                  
##    .Item_32            0.667       NA                  
##    .Item_9             1.293       NA                  
##    .Item_10            1.961       NA                  
##    .Item_11            1.288       NA                  
##    .Item_12            1.342       NA                  
##    .Item_21            0.894       NA                  
##    .Item_22            0.510       NA                  
##    .Item_23            0.338       NA                  
##    .Item_24            1.026       NA                  
##    .Item_33            0.401       NA                  
##    .Item_34            0.532       NA                  
##    .Item_35            0.547       NA                  
##    .Item_36            0.984       NA                  
##     Cognitive          0.593       NA                  
##     Affective          0.639       NA                  
##     Behavioral         0.283       NA                  
##     Absorption         0.000       NA                  
##     Vigor              0.072       NA                  
##     Dedication         0.596       NA

Currently pretty messed up prior to item deletion (Vigor especially). Model also technically not converging need Renata to fix.

Deese guys also do bifactor stuff: Mansolf & Reise (2017)

Summary

Recommendation for final instrument based on consideration of all of the above pieces of evidence

Future plans

Placeholder

Things to do

Research projects

Author bios

Casey Osorio-Duffoo received his M.A. in Industrial and Organizational Psychology from Montclair State University. He enjoys stealing chicken eggs from hen coops and outwitting crows. [NOTE. Currently being held against own will in avian rehab]
Renata Garcia Prieto Palacios Roji is pursuing her Ph.D. in Industrial and Organizational Psychology at Montclair State University. She enjoys allfallfa, the smellll of llavender, and glleefulllly prancing through dew-filllled meadows.
John Kulas received his Ph.D. in Industrial and Organizational Psychology from Northern Illinois University. He feels ashamed of his previous obsession with the evisceration of the mancub Mowgli, and is currently easing into a vegan lifestyle, slowly weaning himself from a diet consisting primarily of foxes and llamas.

References

Placeholder

Spring 2019

Spring 2020

February 4, 2020

February 24, 2020

Structural considerations

Summer 2020

Inter-Item correlations

Scale-level correlations

Substantive scales

Attitudinal scales

Substantive scale (marginal level)

Substantive scale (cell level)

Frequency distributions by dimension

Tables of qualitative indices

Coffman, C., & Harter, J. (1999). A hard look at soft numbers. Position Paper, Gallup Organization.
Cole, M. S., Walter, F., Bedeian, A. G., & O’Boyle, E. H. (2012). Job burnout and employee engagement: A meta-analytic examination of construct proliferation. Journal of Management, 38(5), 1550–1581.
Elloy, D. F., Everett, J. E., & Flynn, W. R. (1991). An examination of the correlates of job involvement. Group & Organization Studies, 16(2), 160–177. https://doi.org/10.1177/105960119101600204
Epskamp, S. (2019). semPlot: Path diagrams and visual analysis of various SEM packages’ output. https://github.com/SachaEpskamp/semPlot
Ferris, R., & Hellier, P. (1984). Added value productivity schemes and employee participation. Asia Pacific Journal of Human Resources, 22(4), 35–44. https://doi.org/10.1177/103841118402200406
Giordano, C., Ones, D. S., Waller, N. G., & Stanek, K. C. (2020). Exploratory bifactor measurement models in vocational behavior research. Journal of Vocational Behavior, 120, 103430. https://doi.org/10.1016/j.jvb.2020.103430
Harter, James K., Schmidt, F., & Hayes, T. (2002). Business-unit-level relationship between employee satisfaction, employee engagement, and business outcomes: A meta-analysis. The Journal of Applied Psychology. https://doi.org/10.1037/0021-9010.87.2.268
Harter, J. K., Schmidt, F. L., Agrawal, S., & Plowman, S. K. (2013). The relationship between engagement at work and organizational outcomes 2012 Q12 meta-analysis lincoln. NE: The Gallup Organization.
Kahn, W. A. (1990). Psychological conditions of personal engagement and disengagement at work. Academy of Management Journal, 33(4), 692–724.
Leiter, M., & Maslach, C. (2004). Areas of worklife: A structured approach to organizational predictors of job burnout. In Research in occupational stress and well-being (Vol. 3, pp. 91–134). https://doi.org/10.1016/S1479-3555(03)03003-8
Leone, D. R. (1995). The relation of work climate, higher order need satisfaction, need salience, and causality orientations to work engagement, psychological adjustment, and job satisfaction [PhD thesis]. ProQuest Information & Learning.
Mansolf, M., & Reise, S. P. (2017). When and why the second-order and bifactor models are distinguishable. Intelligence, 61, 120–129.
Maslach, C., & Leiter, M. (1997). What causes burnout. Maslach C, Leiter MP. The Truth About Burnout: How Organizations Cause Personal Stress and What to Do about It. San Francisco, CA: Josey-Bass Publishers, 38–60.
Maslach, Christina, & Leiter, M. P. (2008). Early predictors of job burnout and engagement. Journal of Applied Psychology, 93(3), 498–512.
Meyer, J. P., & Allen, N. J. (1991). A three-component conceptualization of organizational commitment. Human Resource Management Review, 1(1), 61–89.
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555
Revelle, W. (2021). Psych: Procedures for psychological, psychometric, and personality research. https://personality-project.org/r/psych/ https://personality-project.org/r/psych-manual.pdf
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
Rosseel, Y., Jorgensen, T. D., & Rockwood, N. (2021). Lavaan: Latent variable analysis. https://lavaan.ugent.be
Schaufeli, Wilmar B., & Bakker, A. B. (2003). Utrecht work engagement scale. Occupational Health Psychology Unit Utrecht University, 1.
Schaufeli, W. B., & Bakker, A. B. (2003). UWES–utrecht work engagement scale: Test manual. Unpublished Manuscript: Department of Psychology, Utrecht University, 8.
Schaufeli, Wilmar B., & Bakker, A. B. (2010). Defining and measuring work engagement: Bringing clarity to the concept. Work Engagement: A Handbook of Essential Theory and Research, 12, 10–24.
Schaufeli, Wilmar B., Salanova, M., González-Romá, V., & Bakker, A. B. (2002). The measurement of engagement and burnout: A two sample confirmatory factor analytic approach. Journal of Happiness Studies, 3(1), 71–92.
Schaufeli, W., & Bakker, A. (2010). The conceptualization and measurement of work engagement. In W. Schaufeli, A. Bakker, & M. Leiter (Eds.), Work engagement: A handbook of essential theory and research (pp. 10–24). Psychology Press.
Sievert, C., Parmer, C., Hocking, T., Chamberlain, S., Ram, K., Corvellec, M., & Despouy, P. (2021). Plotly: Create interactive web graphics via plotly.js. https://CRAN.R-project.org/package=plotly
Staw, B. M., Sutton, R. I., & Pelled, L. H. (1994). Employee positive emotion and favorable outcomes at the workplace. Organization Science, 5(1), 51–71.
Taris, T. W., Ybema, J. F., & Beek, I. van. (2017). Burnout and engagement: Identical twins or just close relatives? Burnout Research, 5, 3–11.
Thackray, J. (2005). The gallup Q12. Gallup Management Journal.
Timms, C., Brough, P., & Graham, D. (2012). Burnt-out but engaged: The co-existence of psychological burnout and engagement. Journal of Educational Administration, 50(3), 327–345.